Face Replacement Demo using the Kinect Depth Sensor

This paper proposes a face-swap method, wherein the use of a depth sensor and improved algorithms are used to improve the quality and realism of a face swap process. By tracking head pose and facial features in 3D using a Kinect depth camera, an accurate model of the face can be constructed and used to deform a texture which is then drawn on top of a 2D video stream. The use of random regression forests for head pose estimation is explored, along with a novel method for blending textures based on luminance. The proposed system shows good results through a wider range of head rotations than previous methods, and is able to run at an average of 25fps.